Section: New Results

Situation Awareness

Participants : Christian Laugier, Olivier Simonin, Jilles Dibangoye, David Sierra-Gonzalez, Mathieu Barbier, Victor Romero-Cano, Ozgur Erkent, Christian Wolf.

Dense & Robust outdoor perception for autonomous vehicles

Participants : Christian Laugier, Victor Romero-Cano, Ozgur Erkent, Christian Wolf.

Robust perception plays a crucial role in the development of autonomous vehicles. While perception in normal and constant environmental conditions has reached a plateau, robustly perceiving changing and challenging environments has become an active research topic, particularly due to the safety concerns raised by the introduction of autonomous vehicles to public streets. Solving the robustness issue in road and urban perception applications is the first challenge. Then, it is also mandatory to develop an appropriate framework for extracting relevant semantic information. Our approach is to reason about vision-based data and the output of our grid-based multi-sensors perception approach (see previous section).

The work presented in this section has partly been done in 2016 and completed in 2017, in the scope of our collaboration with Toyota Motor Europe (TME). The main objective was to develop a framework for low-level multi-sensor data fusion in the pixel space. This framework is independent of the scene coverage of any of the sensors. It outputs a new image-like data representation where each pixel contains not only color but also other low level features such as depth and object IDs.

The system depicted by Figure  11 addresses the problem of extending color images with sparse range data at the pixel level. To this end, we developed a framework for densifying sparse range data in the image space. Our framework provides a methodology for creating extended images independently of the density of the range sensor at hand. We adapted and combined two powerful segmentation techniques such as SLIC and Graph Cuts into a hierarchical methodology for depth densification. The experimental results show the advantages of our new low level data representation over using color only in an image classification task. Our framework achieves, from a camera and sparse/cheap range sensors, recognition results that are equivalent to those obtained from a camera and a dense/expensive 3D Velodyne. This work was patented [75] and published in [22].

Novel approach: Towards Semantic Occupancy Grids and robust DATMO(Detection and Tracking of Mobile Objects). Current and future work in the scope of our collaboration with TME, aims at constructing Semantic Occupancy Grids and at developing robust DATMO. The first objective is to develop occupancy grid mapping systems that exploit the precision of Lidar sensors and the dense and high level semantic information that can be obtained from camera sensors, in particular semantic segmentation systems. We are currently developing technologies that allow extending 2D occupancy grids with semantic and appearance information provided by semantic segmentations systems (deep learning based). These technologies allow us to probabilistically fuse occupancy maps and projected 2D segmentation maps both spatially and temporally.

Towards Human-Like Motion Prediction and Decision-Making in Highway Scenarios

Participants : David Sierra-Gonzalez, Christian Laugier, Jilles Dibangoye, Victor Romero-Cano.

The objective of this project is to develop a collision risk estimation system capable of reliably finding the risk of collision associated to the different feasible trajectories of an autonomous or semi-autonomous vehicle. This research work is done in the scope of the Inria-Toyota long-term cooperation and of the PhD thesis work of David Sierra González. The practical output expected for the project is an Advanced Driver Assistance System (ADAS) for the Inria-TME Lexus experimental platform capable of suggesting to the driver in real-time the safest maneuver to perform while driving in the highway.

The research work done in the scope of this project in 2017 has focused on improving the state-of-the-art on short-term prediction of driver behavior in highways. The key idea is to model the behavior of highway drivers from demonstrated highway driving data using Inverse Reinforcement Learning and to exploit the resulting models for prediction. This has led to significant theoretical and experimental results on driver maneuver estimation [25], [26]. The proposed maneuver estimation framework has been validated with real-world highway data collected with an experimental vehicle (see Fig. 12.a). The results show that the system is characterized by short detection times and low false positive rates. This opens the door to the application of this work as part of a collision warning system. In 2017, this work has derived on an additional Inria-TME joint patent [79]. Current efforts are directed towards exploiting the uncertain predictions for decision-making.

The PhD thesis of David Sierra-Gonzalez will be defended in June 2018.

Decision-making for safe road intersection crossing

Participants : Mathieu Barbier, Christian Laugier, Olivier Simonin.

In continuation of the work done in 2016 about the functional space representation [39], we proposed a framework to observe the intention of intruder vehicle approaching an intersection. Random forest classifiers are used in conjunction with our functional discretization to analyze the trajectories of cars approaching an intersection. However each intersection can be different and the approach would scale poorly.

To address this problem, a hybrid data set was constructed. It is built in a simulated environment and completed with real data after a car drove multiple times across the intersection. We compared our approach against other classifiers and space discretization. In addition, we demonstrated the impact and the usefulness of the mixture between simulated and real data. An improvement of 30% accuracy is obtained with the hybrid data set, and 5% using our functional discretization with respect to baseline approach. It was also been implemented in our experimental platform as shown in see Fig. 12.b. This work has been published at IEEE IV 2017 [15].

The PhD thesis of Mathieu Barbier will be defended in June 2018.

The ongoing work use the bayesian network of [56] and extend it to a Partially observable decision process (POMDP). The Observation model uses the performances of the classification described previously to take into account the uncertainty over the intention (This work will be the subject of a paper to be submitted to IEEE IV 2018). The performances of this decision making system is investigated in the scope of the ENABLES-S3 project. With the use of simulation tools, multiple configurations and scenarios are tested. For each test, a set of “Key performances indicator (KPY)” is computed and used to judge the system. Such validation system has yet to be investigated and represent a major interest for both industry and research.